Skip to content

Implement stream connection for remote write #6580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 15, 2025

Conversation

alexqyle
Copy link
Contributor

@alexqyle alexqyle commented Feb 7, 2025

What this PR does:

Introduced stream connection for RemoteWrite endpoint. Keep stream connection open on ingester side could reduce the CPU usage on ingester to open new connection each time accepting new RemoteWrite requests.

Here is the comparison between normal connection and stream connection:

Tested with 6000 TPS

image

Compared to normal connection, stream connection could have 8%-9% less CPU usage.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Alex Le <[email protected]>
@danielblando
Copy link
Contributor

I am ok with this change. Lets get more feedback

@alexqyle alexqyle marked this pull request as ready for review March 4, 2025 19:28
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I like the change. But I don't see any tests here. Can you please add some unit tests and integration tests?

Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we mention this in https://cortexmetrics.io/docs/configuration/v1guarantees/ as experimental feature?

And can we add a simple integration test just to ensure this works E2E?

alexqyle added 2 commits May 14, 2025 15:50
Signed-off-by: Alex Le <[email protected]>
Signed-off-by: Alex Le <[email protected]>
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@yeya24 yeya24 merged commit 5f060b7 into cortexproject:master May 15, 2025
17 checks passed
@alexqyle alexqyle deleted the ingester-stream-push branch May 15, 2025 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants